…native builds)

…y path)

The fast-lio input was pinned to file:///Users/jeffhykin/... which only
exists on the Mac. Repoint to the dimensionalOS/dimos-module-fastlio2
pointlio branch on github so the flake builds on Linux. Same locked rev.

Rename the mirrored fastlio_blueprints.py to pointlio_blueprints.py and
wire it to PointLio (was incorrectly using FastLio2). Adds mid360-pointlio
and mid360-pointlio-voxels to the blueprint registry.

…race-fix flake relock

Adds the offline replay/inspection tooling for the pointlio_native module:
pcap_replay.hpp (streams a Livox pcap into the SDK callbacks), deterministic_clock
+ dual-thread replay options, the ruwik2_pt3 replay harness, the pcap_to_db tool
(append pointlio_odometry into an existing memory db at ~30Hz, streaming), validated
Point-LIO mid360.yaml, and the live smoke-test helper. flake.lock relocks onto the
fastlio2 pointlio rev carrying the mtx_buffer race fix; main.cpp comment corrected
to attribute the divergence fix to that lock (not the publish-rate gating).

Switches the flake's fast-lio input from path:/home/dimos/repos/dimos-module-fastlio2
to github:dimensionalOS/dimos-module-fastlio2/pointlio so the module builds on any
machine without a local fastlio2 clone. Relocked onto rev 7e5d88f (the mtx_buffer
race fix); verified pointlio_native builds from the github source.

…hful config

OUTPUT-model replay of the Jun-7 hand-shake bag diverged to ~25km (vs
hku-mars master's bounded ~5m). Two issues, both fixed here (paired with
the dimos-module-fastlio2 pointlio-branch curvature/deque fixes):

- main.cpp: heap-use-after-free thread race in bag-frame replay. The
  feeder thread drives run_main_iter via step() after each feed, but the
  main thread also drove run_main_iter, so both raced on the shared PCL
  measurement cloud (ASan: free in sync_packages, read-after-free in
  ImuProcess::Process). Force serial_replay in bag-frame mode so only the
  feeder thread touches the EKF.
- CMakeLists.txt + flake.nix: the iVox map backend needs glog; add the
  find_package/link and the nix buildInput. Drop ikd_Tree.cpp from
  sources (iVox replaces ikd-Tree).
- config/mid360.yaml (+ cpp/config duplicate): set satu_acc 5.5->3.0
  (master value — accel >=3g saturated, residual zeroed, keeps velocity
  bounded) and add ivox_grid_resolution=2.0 / ivox_nearby_type=6 (NEARBY6)
  matching the master config that produced the 5.028m baseline.
- replay_ruwik2_pt3.py: make MAX_WALL_SEC env-overridable.

After the fix the OUTPUT model stays bounded (peak |pos| 3.898m) on the
full bag; ASan reports zero memory errors.

…onsumes the iVox fix

The iVox-port + divergence fixes live in dimos-module-fastlio2 @ pointlio
commit 02d5066, which is committed locally but NOT pushed (per the task's
no-push constraint). The flake's fast-lio input was pointing at
github:dimensionalOS/dimos-module-fastlio2/pointlio, whose lock pinned the
pre-fix rev 7e5d88f. Since CMakeLists.txt already drops ikd_Tree.cpp (iVox
replaces ikd-Tree), building against that stale github source fails with
KD_TREE undefined-reference linker errors.

Repoint fast-lio at path:/home/dimos/repos/dimos-module-fastlio2 and bump
the lock so `nix build .#pointlio_native` consumes the local 02d5066 source.
Verified: build exits 0, no KD_TREE errors, 661176-byte binary.

NOTE: this bakes a machine-specific absolute path into the lock. Once
Repo A's pointlio branch is pushed, switch this input back to
github:dimensionalOS/dimos-module-fastlio2/pointlio and bump the lock to
02d5066 for portability.

Now that dimos-module-fastlio2 pointlio (02d5066, the iVox port + divergence
fix) is on origin, switch the fast-lio flake input from the local path: pin
back to github:dimensionalOS/dimos-module-fastlio2/pointlio and bump the lock
to rev 02d5066. This makes jeff/feat/pointlio_native self-contained and
portable (no machine-specific path in the lock).

Verified: nix build .#pointlio_native exits 0, no KD_TREE errors.

… add --force

Extends pcap_to_db.py to populate both pointlio_odometry and pointlio_lidar
streams (start-aligned onto the db's earliest ts) so pointlio can be compared
against fastlio in one recording. --force overwrites existing pointlio streams.
Untracks the machine-specific fastlio_test/setup_network symlinks (enhance
overlay) and drops the empty tools/__init__.py.

…o tools

- Consolidate config/ to one default.yaml (the tuned Mid-360/iVox config);
  remove the unused upstream sensor presets (avia, horizon, marsim, ouster64,
  velodyne, mid360). Module now defaults to default.yaml.
- Remove tools/replay_ruwik2_pt3.py (pcap_to_db.py covers offline replay) and
  tools/demo_live_test.py.

…egistry

- --db is now optional: with no existing db, build one from scratch (defaults
  to <pcap>.db next to the pcap); with an existing db, append + time-align as
  before.
- Rename the internal recorder Rec/RecConfig -> _Rec/_RecConfig so the
  blueprint-registry generator skips it (a tool helper isn't a public module);
  fixes test_all_blueprints_is_current.
- Docstring: document from-scratch generation using the ruwik2_part3 LFS sample.

…yaml

… rm cpp/README + config copies; concise comments

- config: consolidate to a single config/default.yaml; drop the ROS-only
  lid_topic/imu_topic + odom frame-id/publish/pcd_save keys (parsed-but-unused
  by the native binary), and note which params are Mid-360-specific.
- delete cpp/config/{mid360.json,mid360.yaml} (duplicate) and cpp/README.md.
- tighten verbose comments in main.cpp, pcap_replay.hpp, timing.hpp
  (comments only; binary rebuilds clean).

… db name in docstring

120s clip (elapsed 55-175s) of the ruwik velocity-spike recording — the data
that diverges through FAST-LIO at the 0.5m pre-KF voxel and is bounded under
Point-LIO. Use via get_data('ruwik2_part3'); pcap_to_db --pcap <it> builds the
db from scratch.

…rt3 sample

….0 clock bug

- module.py docstring: Point-LIO (not FAST-LIO2) + correct import path.
- pcap_to_db.py: use 'is not None' (not 'or') for the ts fallback so a real
  sensor ts of 0.0 isn't replaced by wall time (which would misclassify the
  stream clock in _resolve_offset).

… NIC

Reads a Livox Mid-360 pcap into RAM, rewrites packet timestamps to
current-time, and replays point/IMU/status onto a virtual network
interface at a configurable rate + delay. Synthesizes the Livox SDK2
control protocol (discovery + GetInternalInfo/FwType ACKs, CRC16/CRC32)
so an unmodified consumer (pointlio) handshakes with it as a real sensor.
Builds via nix (rustPlatform.buildRustPackage, cargoLock from Cargo.lock).

…bal_map

- Cloud now published in the sensor frame (mid360_link): use fastlio2
  get_body_cloud() (undistorted scan, no world registration) instead of
  inverse-transforming the world cloud. No transform in publish_lidar.
- Split frames: frame_id (mid360_link) on both the cloud + odometry
  headers; odom_parent_frame_id (odom) -> odom_frame_id (base_link) for
  the TF publish.
- Remove global_map / voxel_map.hpp entirely (deleted file, config,
  blueprint + pcap_to_db references).
- Bump fast-lio pin to fcbd1c2 (adds get_body_cloud).

…gister

get_data() at module level triggered a git-LFS download during blueprint
validation (test_blueprint_is_valid), which CI blocks via git-lfs-guard —
failing the whole test matrix. Default pcap to empty; resolve the capture
path at run time instead.

Port the minimal pcap-replay subsystem from jeff/feat/go2_record into the
clean branch so FAST-LIO can run offline from a Mid-360 pcap, matching the
Point-LIO pcap_to_db workflow:

- cpp: pcap_replay.hpp + timing.hpp (header-only), main.cpp refactored so the
  main loop runs from either the live SDK or a pcap feeder thread, with an
  optional deterministic sensor-clock mode. Keeps the clean branch's
  velocity-cap (guarded set_max_velocity_norm_ms) and flake (velocity-cap
  fast-lio); does not pull go2_record's tcpdump record path.
- module.py: replay_pcap / replay_skip_until_ns / first_packet_marker /
  deterministic_clock config fields; skip network validation in replay mode.
- tools/pcap_to_db.py: replay a pcap through FastLio2 (real-time, non-
  deterministic) and append fastlio_odometry + fastlio_lidar into an existing
  memory2 db, time-aligned onto its clock. --force overwrites.

…FastLio2Recorder

- livox/pcap_recorder.py: standalone tcpdump pcap capture (LivoxPcapRecorder),
  decoupled from FAST-LIO. The lidar/SLAM module no longer owns packet capture.
- fastlio2/recorder.py: FastLio2Recorder records fastlio_odometry + fastlio_lidar
  and rewrites ONLY those streams' timestamps onto the db clock (promoted from
  pcap_to_db's inline recorder; fixes the ts==0.0 falsy-fallback bug).
- pcap_to_db.py now imports FastLio2Recorder instead of an inline copy.

FastLio2 no longer produces a global voxel map — odometry + registered lidar
only. Removed global_map Out, map config, and the mapping.GlobalPointcloud spec.
Updated all consumers (fastlio_blueprints, alfred_nav, g1_onboard, g1_nav_onboard,
mobile, pcap_to_db) to drop map args + the global_map remap. Nav blueprints lose
their fastlio map; full nav wants a separate mapper wired in (follow-up).

The pose-setter dict is typed dict[str, Any] (to avoid evaluating the Pose
forward-ref at class-definition time), so cast its result back to Pose | None.

The pointlio improvements (no-yaml config, Recorder-subclass rework,
frame_id scheme) move to jeff/feat/pointlio_native via merge; this PR
carries only the fastlio2 work + shared memory2 Recorder base.

…-only)"

This reverts commit 0260030.

# Conflicts:
#	dimos/hardware/sensors/lidar/fastlio2/tools/pcap_to_db.py
#	dimos/memory2/module.py

…tive

# Conflicts:
#	dimos/hardware/sensors/lidar/pointlio/module.py

Convert the memory2 Recorder from thread/disposable rx subscriptions to
manual async callbacks via process_observable, and let pose_setter_for
methods be async (awaited in _resolve_pose). Update the fastlio and go2
recorders accordingly.

Raise TypeError at decoration time if a non-async function is decorated,
and always await the setter in _resolve_pose.

Switch recorders to async: the base Recorder now uses process_observable
async callbacks and pose_setter_for requires async setters. Converted the
pointlio recorder's pose setters to async to match.

…pointlio_native

… config

…tive

Stamp every output as (packet_sensor_time + offset), with offset locked at
the first drain, instead of system_clock::now(). Output timestamps now follow
the data's real cadence, so a fast replay's odom span reaches the pcap length
in proportionally less walltime (no-op on live hardware, where sensor time
already advances in real time). Stop the replay ~1s after the odometry stream
goes stagnant (the pcap has drained) rather than at a fixed wall-clock cap.

Add an overload guard in the drain loop: each frame's swallowed sensor-time
span over the wall interval is how much faster than real-time packets are
arriving; when that exceeds 1.5x it logs a throttled warning (one line per 10s
reporting the window's peak, surfacing the UDP-drop/drift risk that heavy load
or fast replay causes.

pcap_to_db: --run-name (suffix the stream), --no-lidar (skip the bulky lidar
stream), --replace, and a guard that refuses to overwrite an existing stream.
Recorder: skip_ports to leave selected in-ports unrecorded.
EOF
)

Lead with the user-actionable message ("high risk of odom drift … Try
reducing CPU load") and keep the real-time ratio as a compact diagnostic tail.

…ndent replay

Frame-cut and lidar/odom emission were gated on the wall clock (steady_clock),
so a faster --rate packed more sensor data into each fixed wall-time frame. That
changed the EKF's input stream with replay speed: a 2x replay drifted differently
from the 1x live run it was meant to reproduce.

Drive the emission gates off a data clock (the newest packet's sensor timestamp,
now atomic instead. A frame always spans exactly one frame_interval of data on a
fixed grid anchored at the first packet, so 1x and 2x hit identical frame/odom
boundaries and produce identical geometry. On live hardware packet time tracks
wall time, so this is a no-op there (one code path, no flag). The overload check
keeps its own wall-time bookmark, since it is inherently about real elapsed time.

Verified on china_office1: 1x vs 2x now match (835.45m vs 835.21m path,
loop-error 6.439m vs 6.452m, trajectory RMS 0.106m over an 835m path); previously
2x produced ~14.3k odom rows vs ~33.5k at 1x and diverged.
EOF
)

PointLio already rebroadcasts its odometry as the moving odom->body tf edge,
but pcap_to_db ran the recorder with record_tf=False, so no tf was ever saved
and the only remaining edges (map->odom root, body->base_link bridge, Go2/Mid-360
sensor mounts) had no publisher at all.

Add Go2Mid360ReplayStaticTf: a module that subscribes to the odometry stream and
re-stamps the fixed edges at each odom's data-time ts (throttled), then flip the
recorder to record_tf=True. Driving the static edges off the odometry data clock
(rather than the stock StaticTfPublisher's wall-clock publish loop keeps them
aligned with PointLio's stamps under rate-scaled replay, so the recorded tree
stays connected across the whole trajectory instead of only the wall-clock span.
EOF
)